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CLAIMS 

What is claimed is: 

1 . A query system for structured multimedia content retrieval, said system including: 

a query language having query constructs and formalisms for specifying characteristics of 
extensible markup language (XML) documents for retrieval; and 

wherein said characteristics include spatial, temporal, and visual datatypes. 

2. A query system as recited in claim 1, wherein said query language includes: 
means for resolving intensional data and relationships arising from any of: 

(a) XML datatype mechanism; 

(b) irregular XML structures; and 

(c) co-occurrence constraints. 

3. A query system as recited in claim 2, wherein said means for resolving comprises a logic 
formalism for supporting queries on XML documents with any of: 

(A) intensional data and relationships; 

(B) irregular document structures; and 

(C) and co-occurrence constraints. 

4. A query system as recited in claim 1 , wherein said query language includes 

means for identifying specification issues in XML query language for XML document 
retrieval. 
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5. A query system as recited in claim 1, wherein said query language includes 

means for identifying specification issues in XML query language for MPEG-7 
document retrieval. 

6. A query system for structured multimedia content retrieval, said system including: 

a query language based on logic formalism for content retrieval; and 

said logic formalism including atomic logic formulas, said atomic logic formulas being 
element predicates in a relational calculus. 

7. A query system as recited in claim 6, wherein said query language comprises query 
constructs and formalisms for specifying different aspects of extensible markup language (XML) 
documents. 

8. A query system as recited in claim 7, wherein said query constructs and formalisms are 
adapted for spatial, temporal and visual datatypes. 

9. A query system as recited in claim 7, wherein said query constructs and formalisms are 
adapted for spatial, temporal and visual datatypes in MPEG-7 documents. 

10. A query system as recited in claim 6, wherein: 

queries in said relational calculus are equivalent to a proof- finding process; and 

said proof-finding process comprises finding all proofs to existential closure of logical 
assertions in the form of path predicates required to be satisfied by tree document elements. 

11. A query system as recited in claim 7, wherein spatial, temporal and visual datatypes and 
relationships are described in said logic formalism for content retrieval. 
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12. A query system as recited in claim 7, wherein said query language includes: 
means for resolving intensional data and relationships arising from any of: 

(a) XML datatype mechanism; 

(b) irregular XML structures; and 
5 (c) co-occurrence constraints. 

13. A query system as recited in claim 12, wherein said query language includes 

means for identifying specification issues in XML query language for XML document 
retrieval. 

14. A query system as recited in claim 1 3, wherein said query language includes 

means for identifying specification issues in XML query language for MPEG-7 
document retrieval. 

15. A query system for structured multimedia content retrieval, said system including: 

a query language based on logic formalism for content retrieval; and 

said logic formalism including atomic logic formulas, said atomic logic formulas being 
element predicates in a relational calculus. 

1 6. A query system as recited in claim 15, wherein: 

queries in said relational calculus are equivalent to a proof- finding process; and 

said proof-finding process comprises finding all proofs to existential closure of logical 
assertions in the form of path predicates required to be satisfied by tree document elements. 
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17. A query system as recited in claim 15, wherein spatial, temporal and visual datatypes and 
relationships are described in said logic formalism for content retrieval. 

18. A query system as recited in claim 15, wherein said query language includes: 
means for resolving intensional data and relationships arising from any of: 

(a) XML datatype mechanism; 

(b) irregular XML structures; and 
5 (c) co-occurrence constraints. 

19. A query system as recited in claim 18, wherein said query language includes 

means for identifying specification issues in XML query language for XML document 
retrieval. 

20. A query system as recited in claim 19, wherein said query language includes 

means for identifying specification issues in XML query language for MPEG-7 
document retrieval. 

21 . A query system for structured multimedia content retrieval, said system including: 
a query language based on logic formalism for content retrieval; 

said logic formalism including atomic logic formulas, said atomic logic formulas being 
element predicates in a relational calculus; 

5 means for identifying given specifications of multimedia XML documents in MPEG-7 

XML query specifications; and 

means for applying said logic formalism for processing said given specifications for 
specifying spatial and temporal relationships pertaining to said XML documents to support 
MPEG-7 XML document retrieval and modification of multimedia XML documents. 
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22. A query system in accordance with claim 21, wherein said given specifications include 
intensional data and relationships specifications, document addressing specifications, and co- 
occurrence constraints specifications. 

23. A query system in accordance with claim 21, wherein said given specifications include 
element datatypes. 

24. A query system in accordance with claim 23, wherein spatial and temporal relationships 
are derived from said element datatypes. 

25. A query system in accordance with claim 24, wherein spatial and temporal relationships 
are further included in said given specifications as a complex datatype for multimedia XML 
documents. 

26. A query system in accordance with claim 23, wherein said datatypes.include: 

(A) Mpeg7Type, basic datatypes, reference datatypes, unique identifier datatypes, and 
time datatypes; 

5 (B) MPEG-7 visual datatypes used to specify visual properties of multimedia objects, 

including spatial, color, texture, motion, location; and 

(C) MPEG-7 audio datatypes are used to specify audio content. 
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27. A query system in accordance with claim 2 1 , including a tool for generating a description 
from a video based on a scene change technique, said tool including processing means for: 

(a) breaking down the video temporally into scenes or shots using scene change 
detection algorithms that can detect both abrupt as well as gradual changes; 

5 (b) outlining user-identified objects of interest within said scenes; 

(c) tracking said user-identified objects; 

(d) creating a node point where a significant motion change wherein a linear mode is 
inadequate; 

(e) providing the specification of said user-identified objects as any of temporal, 
10 audio, and visual datatypes; and 

(f) providing a description of said user-defined objects as any of spatial, temporal 
and visual datatypes. 

28. A query system in accordance with claim 27, wherein processing means provides said 
tracking said user-identified objects in a semi-automatic manner. 

29. A query system in accordance with claim 27, wherein processing means provides said 
description of said user-defined objects said by the use of abstract datatype techniques (ADT). 



30. A query system in accordance with claim 27, wherein processing means provides said 
respective datatypes as composite datatypes constructed from more primitive ones. 



31. A query system for multimedia content retrieval, said system including: 
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a query language based on logic formalism for content retrieval, said logic formalism 
being hereinafter referred to as Path Predicate Calculus and being utilized for logic-based queries 
and manipulations; 

said Path Predicate Calculus including atomic logic formulas, said atomic logic formulas 
being element predicates in a relational calculus and comprising element predicates and path 
predicates, for asserting logical truth statements about document elements in a document tree; 

means for identifying given specifications of multimedia XML documents in MPEG-7 
XML query specifications; and 

means for applying said logic formalism for processing said given specifications for 
specifying spatial and temporal relationships pertaining to said XML documents to support 
MPEG-7 XML document retrieval and modification, multimedia XML documents. 

32. A query system as recited in claim 3 1 , wherein: 

queries in said relational calculus are equivalent to a proof- finding process; and 

said proof-finding process comprises finding all proofs to existential closure of logical 
assertions in the form of path predicates required to be satisfied by tree document elements. 

33. A query system as recited in claim 3 1 , wherein spatial, temporal and visual datatypes and 
relationships are described in said logic formalism for content retrieval. 

34. A query system as recited in claim 3 1 , wherein said query language includes: 
means for resolving intensional data and relationships arising from any of: 

(a) XML datatype mechanism; 

(b) irregular XML structures; and 

(c) co-occurrence constraints. 
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35. A query system as recited in claim 34, wherein said query language includes 

means for identifying specification issues in XML query language for XML document 
retrieval. 

36. A query system for structured multimedia content retrieval, said system including: 

a query language based on logic formalism for content retrieval, said language including 
query constructs and formalisms for specifying different aspects of XML documents; and 

wherein said constructs and formalisms are particularly adapted for spatial, temporal and 
5 visual datatypes. 

37. A query system as recited in claim 36, wherein said query language identifies intensional 
data and relationships due to XML datatype mechanisms, irregular XML structures, and co- 
occurrence constraints for document retrieval. 

38. A query system as recited in claim 37, wherein said query language is specially adapted 
for MPEG-7 documents. 

39. A method for for structured multimedia content retrieval, said method comprising: 

utilizing a query language based on logic formalism for content retrieval, said logic 
formalism including atomic logic formulas, said atomic logic formulas being element predicates 
in a relational calculus; 

5 identifying given specifications of multimedia XML documents in MPEG-7 XML query 

specifications; and 

applying said logic formalism for processing said given specifications for specifying 
spatial and temporal relationships pertaining to said XML documents to support MPEG-7 XML 
document retrieval and modification of multimedia XML documents. 
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40. A method as recited in claim 39, comprising: 

generating a description from a video based on a scene change technique, said generating 
including the steps of: 

(a) breaking down the video temporally into scenes or shots using scene change 
5 detection algorithms that can detect both abrupt as well as gradual changes; 

(b) outlining user-identified objects of interest within said scenes; 

(c) tracking said user-identified objects; 

(d) creating a node point where a significant motion change wherein a linear mode is 
inadequate; 

10 (e) providing the specification of said user-identified objects as any of temporal, 

audio, and visual datatypes; and 

(f) providing a description of said user-defined objects as any of spatial, temporal 
and visual datatypes. 



